18 research outputs found
Hessian-Free High-Resolution Nesterov Acceleration for Sampling
We propose an accelerated-gradient-based MCMC method. It relies on a
modification of the Nesterov's accelerated gradient method for strongly convex
functions (NAG-SC): We first reformulate NAG-SC as a Hessian-Free
High-Resolution ODE, then release the high-resolution coefficient as a free
hyperparameter, and finally inject appropriate noise and discretize the
diffusion process. Accelerated sampling enabled by this new hyperparameter is
not only experimentally demonstrated on several learning tasks, but also
theoretically quantified, both at the continuous level and after
discretization. For (not-necessarily-strongly-) convex and -smooth
potentials, exponential convergence in divergence is proved, with a
rate analogous to state-of-the-art results of underdamped Langevin dynamics,
plus an additional acceleration. At the same time, the method also works for
nonconvex potentials, for which we also establish exponential convergence as
long as the potential satisfies a Poincar\'e inequality
NySALT: Nystr\"{o}m-type inference-based schemes adaptive to large time-stepping
Large time-stepping is important for efficient long-time simulations of
deterministic and stochastic Hamiltonian dynamical systems. Conventional
structure-preserving integrators, while being successful for generic systems,
have limited tolerance to time step size due to stability and accuracy
constraints. We propose to use data to innovate classical integrators so that
they can be adaptive to large time-stepping and are tailored to each specific
system. In particular, we introduce NySALT, Nystr\"{o}m-type inference-based
schemes adaptive to large time-stepping. The NySALT has optimal parameters for
each time step learnt from data by minimizing the one-step prediction error.
Thus, it is tailored for each time step size and the specific system to achieve
optimal performance and tolerate large time-stepping in an adaptive fashion. We
prove and numerically verify the convergence of the estimators as data size
increases. Furthermore, analysis and numerical tests on the deterministic and
stochastic Fermi-Pasta-Ulam (FPU) models show that NySALT enlarges the maximal
admissible step size of linear stability, and quadruples the time step size of
the St\"{o}rmer--Verlet and the BAOAB when maintaining similar levels of
accuracy.Comment: 26 pages, 7 figure
Doubly Robust Augmented Model Accuracy Transfer Inference with High Dimensional Features
Due to label scarcity and covariate shift happening frequently in real-world
studies, transfer learning has become an essential technique to train models
generalizable to some target populations using existing labeled source data.
Most existing transfer learning research has been focused on model estimation,
while there is a paucity of literature on transfer inference for model accuracy
despite its importance. We propose a novel oubly obust
ugmented odel ccuracy ransfer
nferene (DRAMATIC) method for point and interval
estimation of commonly used classification performance measures in an unlabeled
target population using labeled source data. Specifically, DRAMATIC derives and
evaluates the risk model for a binary response against some low dimensional
predictors on the target population, leveraging from source
data only and high dimensional adjustment features from both the
source and target data. The proposed estimators are doubly robust in the sense
that they are consistent when at least one model is correctly
specified and certain model sparsity assumptions hold. Simulation results
demonstrate that the point estimation have negligible bias and the confidence
intervals derived by DRAMATIC attain satisfactory empirical coverage levels. We
further illustrate the utility of our method to transfer the genetic risk
prediction model and its accuracy evaluation for type II diabetes across two
patient cohorts in Mass General Brigham (MGB) collected using different
sampling mechanisms and at different time points
Non-intrusive and structure preserving multiscale integration of stiff ODEs, SDEs and Hamiltonian systems with hidden slow dynamics via flow averaging
We introduce a new class of integrators for stiff ODEs as well as SDEs. These
integrators are (i) {\it Multiscale}: they are based on flow averaging and so
do not fully resolve the fast variables and have a computational cost
determined by slow variables (ii) {\it Versatile}: the method is based on
averaging the flows of the given dynamical system (which may have hidden slow
and fast processes) instead of averaging the instantaneous drift of assumed
separated slow and fast processes. This bypasses the need for identifying
explicitly (or numerically) the slow or fast variables (iii) {\it
Nonintrusive}: A pre-existing numerical scheme resolving the microscopic time
scale can be used as a black box and easily turned into one of the integrators
in this paper by turning the large coefficients on over a microscopic timescale
and off during a mesoscopic timescale (iv) {\it Convergent over two scales}:
strongly over slow processes and in the sense of measures over fast ones. We
introduce the related notion of two-scale flow convergence and analyze the
convergence of these integrators under the induced topology (v) {\it Structure
preserving}: for stiff Hamiltonian systems (possibly on manifolds), they can be
made to be symplectic, time-reversible, and symmetry preserving (symmetries are
group actions that leave the system invariant) in all variables. They are
explicit and applicable to arbitrary stiff potentials (that need not be
quadratic). Their application to the Fermi-Pasta-Ulam problems shows accuracy
and stability over four orders of magnitude of time scales. For stiff Langevin
equations, they are symmetry preserving, time-reversible and Boltzmann-Gibbs
reversible, quasi-symplectic on all variables and conformally symplectic with
isotropic friction.Comment: 69 pages, 21 figure
Spatial distribution and potential sources of arsenic and water-soluble ions in the snow at Ili River Valley, China
Trace elements and water-soluble ions in snow can be used as indicators to reveal natural and anthropogenic emissions. To understand the chemical composition, characteristics of snow and their potential sources in the Ili River Valley (IRV), snow samples were collected from 17 sites in the IRV from December 2018 to March 2019. Inverse distance weighting, enrichment factor (EF) analysis, and backward trajectory modelling were applied to evaluate the spatial distributions and sources of water-soluble ions and dissolved arsenic (As) in snow. The re-sults indicate that Ca2+ and SO42-were the dominant ions, and the concentrations of As ranged from 0.09 to 0.503 mu g L-1. High concentrations of As were distributed in the northwest and middle of the IRV, and the concentrations of the major ions were high in the west of the IRV. The strong correlation of As with F-, SO42-, and NO2- demonstrates that As mainly originated from coal-burning and agricultural activities. Principal component analysis showed that the ions originated from a combination of anthropogenic and crustal sources. The EFs showed that K+, SO42-, and Mg2+ were mainly influenced by human activities. Backward trajectory cluster analysis suggested that the chemical composition of snow was affected by soil dust transport from the western air mass, the unique terrain, and local anthropogenic activities. These results provide important sci-entific insights for atmospheric environmental management and agricultural production within the IRV